Prediction of Effectiveness and Toxicities of Immune Checkpoint Inhibitors Using Real-World Patient Data.

PURPOSE: Although immune checkpoint inhibitors (ICIs) have improved outcomes in certain patients with cancer, they can also cause life-threatening immunotoxicities. Predicting immunotoxicity risks alongside response could provide a personalized risk-benefit profile, inform therapeutic decision making, and improve clinical trial cohort selection. We aimed to build a machine learning (ML) framework using routine electronic health record (EHR) data to predict hepatitis, colitis, pneumonitis, and 1-year overall survival. METHODS: Real-world EHR data of more than 2,200 patients treated with ICI through December 31, 2018, were used to develop predictive models. Using a prediction time point of ICI initiation, a 1-year prediction time window was applied to create binary labels for the four outcomes for each patient. Feature engineering involved aggregating laboratory measurements over appropriate time windows (60-365 days). Patients were randomly partitioned into training (80%) and test (20%) sets. Random forest classifiers were developed using a rigorous model development framework. RESULTS: The patient cohort had a median age of 63 years and was 61.8% male. Patients predominantly had melanoma (37.8%), lung cancer (27.3%), or genitourinary cancer (16.4%). They were treated with PD-1 (60.4%), PD-L1 (9.0%), and CTLA-4 (19.7%) ICIs. Our models demonstrate reasonably strong performance, with AUCs of 0.739, 0.729, 0.755, and 0.752 for the pneumonitis, hepatitis, colitis, and 1-year overall survival models, respectively. Each model relies on an outcome-specific feature set, though some features are shared among models. CONCLUSION: To our knowledge, this is the first ML solution that assesses individual ICI risk-benefit profiles based predominantly on routine structured EHR data. As such, use of our ML solution will not require additional data collection or documentation in the clinic.

Identification and Characterization of Avoidable Hospital Admissions in Patients With Lung Cancer.

BACKGROUND: More than 50% of patients with lung cancer are admitted to the hospital while receiving treatment, which is a burden to patients and the healthcare system. This study characterizes the risk factors and outcomes of patients with lung cancer who were admitted to the hospital. METHODS: A multidisciplinary oncology care team conducted a retrospective medical record review of patients with lung cancer admitted in 2018. Demographics, disease and admission characteristics, and end-of-life care utilization were recorded. Following a multidisciplinary consensus review process, admissions were determined to be either "avoidable" or "unavoidable." Generalized estimating equation logistic regression models assessed risks and outcomes associated with avoidable admissions. RESULTS: In all, 319 admissions for 188 patients with a median age of 66 years (IQR, 59-74 years) were included. Cancer-related symptoms accounted for 65% of hospitalizations. Common causes of unavoidable hospitalizations were unexpected disease progression causing symptoms, chronic obstructive pulmonary disease exacerbation, and infection. Of the 47 hospitalizations identified as avoidable (15%), the median overall survival was 1.6 months compared with 9.7 months (hazard ratio, 2.07; 95% CI, 1.34-3.19; P<.001) for unavoidable hospitalizations. Significant reasons for avoidable admissions included cancer-related pain (P=.02), hypervolemia (P=.03), patient desire to initiate hospice services (P=.01), and errors in medication reconciliation or distribution (P<.001). Errors in medication management caused 26% of the avoidable hospitalizations. Of admissions in patients receiving immunotherapy (n=102) or targeted therapy (n=44), 9% were due to adverse effects of treatment. Patients receiving immunotherapy and targeted therapy were at similar risk of avoidable hospitalizations compared with patients not receiving treatment (P=.3 and P=.1, respectively). CONCLUSIONS: We found that 15% of hospitalizations among patients with lung cancer were potentially avoidable. Uncontrolled symptoms, delayed implementation of end-of-life care, and errors in medication reconciliation were associated with avoidable inpatient admissions. Symptom management tools, palliative care integration, and medication reconciliations may mitigate hospitalization risk.

Accuracy and Reliability of Chatbot Responses to Physician Questions.

Importance: Natural language processing tools, such as ChatGPT (generative pretrained transformer, hereafter referred to as chatbot), have the potential to radically enhance the accessibility of medical information for health professionals and patients. Assessing the safety and efficacy of these tools in answering physician-generated questions is critical to determining their suitability in clinical settings, facilitating complex decision-making, and optimizing health care efficiency. Objective: To assess the accuracy and comprehensiveness of chatbot-generated responses to physician-developed medical queries, highlighting the reliability and limitations of artificial intelligence-generated medical information. Design, Setting, and Participants: Thirty-three physicians across 17 specialties generated 284 medical questions that they subjectively classified as easy, medium, or hard with either binary (yes or no) or descriptive answers. The physicians then graded the chatbot-generated answers to these questions for accuracy (6-point Likert scale with 1 being completely incorrect and 6 being completely correct) and completeness (3-point Likert scale, with 1 being incomplete and 3 being complete plus additional context). Scores were summarized with descriptive statistics and compared using the Mann-Whitney U test or the Kruskal-Wallis test. The study (including data analysis) was conducted from January to May 2023. Main Outcomes and Measures: Accuracy, completeness, and consistency over time and between 2 different versions (GPT-3.5 and GPT-4) of chatbot-generated medical responses. Results: Across all questions (n = 284) generated by 33 physicians (31 faculty members and 2 recent graduates from residency or fellowship programs) across 17 specialties, the median accuracy score was 5.5 (IQR, 4.0-6.0) (between almost completely and complete correct) with a mean (SD) score of 4.8 (1.6) (between mostly and almost completely correct). The median completeness score was 3.0 (IQR, 2.0-3.0) (complete and comprehensive) with a mean (SD) score of 2.5 (0.7). For questions rated easy, medium, and hard, the median accuracy scores were 6.0 (IQR, 5.0-6.0), 5.5 (IQR, 5.0-6.0), and 5.0 (IQR, 4.0-6.0), respectively (mean [SD] scores were 5.0 [1.5], 4.7 [1.7], and 4.6 [1.6], respectively; P = .05). Accuracy scores for binary and descriptive questions were similar (median score, 6.0 [IQR, 4.0-6.0] vs 5.0 [IQR, 3.4-6.0]; mean [SD] score, 4.9 [1.6] vs 4.7 [1.6]; P = .07). Of 36 questions with scores of 1.0 to 2.0, 34 were requeried or regraded 8 to 17 days later with substantial improvement (median score 2.0 [IQR, 1.0-3.0] vs 4.0 [IQR, 2.0-5.3]; P < .01). A subset of questions, regardless of initial scores (version 3.5), were regenerated and rescored using version 4 with improvement (mean accuracy [SD] score, 5.2 [1.5] vs 5.7 [0.8]; median score, 6.0 [IQR, 5.0-6.0] for original and 6.0 [IQR, 6.0-6.0] for rescored; P = .002). Conclusions and Relevance: In this cross-sectional study, chatbot generated largely accurate information to diverse medical queries as judged by academic physician specialists with improvement over time, although it had important limitations. Further research and model development are needed to correct inaccuracies and for validation.

Implementing Innovation: Informatics-Based Technologies to Improve Care Delivery and Clinical Research.

Improving technology has promised to improved health care delivery and the lives of patients. The realized benefits of technology, however, are delayed or less than anticipated. Three recent technology initiatives are reviewed: the Clinical Trials Rapid Activation Consortium (CTRAC), minimal Common Oncology Data Elements (mCODE), and electronic Patient-Reported Outcomes. Each initiative is at a different stage of maturity but promises to improve the delivery of cancer care. CTRAC is an ambitious initiative funded by the National Cancer Institute (NCI) to develop processes across multiple NCI-supported cancer centers to facilitate the development of centralized electronic health record (EHR) treatment plans. Facilitating interoperability of treatment regimens has the potential to improve sharing between centers and decrease the time to begin clinical trials. The mCODE initiative began in 2019 and is currently Standard for Trial Use version 2. This data standard provides an abstraction layer on top of EHR data and has been implemented across more than 60 organizations. Patient-reported outcomes have been shown to improve patient care in numerous studies. Best practices for how to leverage these in an oncology practice continue to evolve. These three examples show how innovative has diffused into practice and evolved cancer care delivery and highlight a movement toward patient-centered data and interoperability.

A Unified Approach to Clinical Informatics Education for Undergraduate and Graduate Medical Education.

Vanderbilt University Medical Center has adopted a unified approach to undergraduate and graduate clinical informatics education. Twenty-three learners have completed the course which is designed around four key activities: 1) didactic sessions 2) informatics history and physical where learners observe clinical areas, document workflows, identify a problem to solve and propose an informatics-informed solution 3) informatics clinic where learners are side-by-side with practicing clinical informaticians and 4) interactive learning activities where student groups work through case-based informatics problems with an informatics preceptor. These experiences are coupled with opportunities for asynchronous projects, reflections, and weekly assessments. The curriculum learning objectives are modeled after the clinical informatics fellowship curriculum. Feedback suggests the course is achieving the planned goals. It is a feasible model for other institutions and addresses knowledge gaps in clinical informatics for undergraduate and graduate medical education learners.

Follow-up Interactive Long-Term Expert Ranking (FILTER): a crowdsourcing platform to adjudicate risk for survivorship care.

OBJECTIVES: To develop an online crowdsourcing platform where oncologists and other survivorship experts can adjudicate risk for complications in follow-up. MATERIALS AND METHODS: This platform, called Follow-up Interactive Long-Term Expert Ranking (FILTER), prompts participants to adjudicate risk between each of a series of pairs of synthetic cases. The Elo ranking algorithm is used to assign relative risk to each synthetic case. RESULTS: The FILTER application is currently live and implemented as a web application deployed on the cloud. DISCUSSION: While guidelines for following cancer survivors exist, refinement of survivorship care based on risk for complications after active treatment could improve both allocation of resources and individual outcomes in long-term follow-up. CONCLUSION: FILTER provides a means for a large number of experts to adjudicate risk for survivorship complications with a low barrier of entry.

Women screened for breast cancer are dying from lung cancer: An opportunity to improve lung cancer screening in a mammography population.

OBJECTIVE: Lung cancer is the leading cancer killer in women, resulting in more deaths than breast, cervical and ovarian cancer combined. Screening for lung cancer has been shown to significantly reduce mortality, with some evidence that women may have a greater benefit. This study demonstrates that a population of women being screened for breast cancer may greatly benefit from screening for lung cancer. METHODS: Data from 18,040 women who were screened for breast cancer in 2015 at two imaging facilities that also performed lung screening were reviewed. A natural language-processing algorithm followed by a manual chart review identified women eligible for lung cancer screening by U.S. Preventive Services Task Force (USPSTF) criteria. A chart review of these eligible women was performed to determine subsequent enrollment in a lung screening program (2016-2019), current screening eligibility, cancer diagnoses and cancer-related outcomes. RESULTS: Natural language processing identified 685 women undergoing screening mammography who were also potentially eligible for lung screening based on age and smoking history. Manual chart review confirmed 251 were eligible under USPSTF criteria. By June 2019, 63 (25%) had enrolled in lung screening, of which three were diagnosed with screening-detected lung cancer resulting in zero deaths. Of 188 not screened, seven were diagnosed with lung cancer resulting in five deaths by study end. Four women received a diagnosis of breast cancer with no deaths. CONCLUSION: Women screened for breast cancer are dying from lung cancer. We must capitalize on reducing barriers to improve screening for lung cancer among high-risk women.

Learnings From Precision Clinical Trial Matching for Oncology Patients Who Received NGS Testing.

PURPOSE: Tumor next-generation sequencing reports typically generate trial recommendations for patients based on their diagnosis and genomic profile. However, these require additional refinement and prescreening, which can add to physician burden. We wanted to use human prescreening efforts to efficiently refine these trial options and also elucidate the high-value parameters that have a major impact on efficient trial matching. METHODS: Clinical trial recommendations were generated based on diagnosis and biomarker criteria using an informatics platform and were further refined by manual prescreening. The refined results were then compared with the initial trial recommendations and the reasons for false-positive matches were evaluated. RESULTS: Manual prescreening significantly reduced the number of false positives from the informatics generated trial recommendations, as expected. We found that trial-specific criteria, especially recruiting status for individual trial arms, were a high value parameter and led to the largest number of automated false-positive matches. CONCLUSION: Reflex clinical trial matching approaches that refine trial recommendations based on the clinical details as well as trial-specific criteria have the potential to help alleviate physician burden for selecting the most appropriate trial for their patient. Investing in publicly available resources that capture the recruiting status of a trial at the cohort or arm level would, therefore, allow us to make meaningful contributions to increase the clinical trial enrollments by eliminating false positives.

A retrospective approach to evaluating potential adverse outcomes associated with delay of procedures for cardiovascular and cancer-related diagnoses in the context of COVID-19.

OBJECTIVE: During the COVID-19 pandemic, health systems postponed non-essential medical procedures to accommodate surge of critically-ill patients. The long-term consequences of delaying procedures in response to COVID-19 remains unknown. We developed a high-throughput approach to understand the impact of delaying procedures on patient health outcomes using electronic health record (EHR) data. MATERIALS AND METHODS: We used EHR data from Vanderbilt University Medical Center's (VUMC) Research and Synthetic Derivatives. Elective procedures and non-urgent visits were suspended at VUMC between March 18, 2020 and April 24, 2020. Surgical procedure data from this period were compared to a similar timeframe in 2019. Potential adverse impact of delay in cardiovascular and cancer-related procedures was evaluated using EHR data collected from January 1, 1993 to March 17, 2020. For surgical procedure delay, outcomes included length of hospitalization (days), mortality during hospitalization, and readmission within six months. For screening procedure delay, outcomes included 5-year survival and cancer stage at diagnosis. RESULTS: We identified 416 surgical procedures that were negatively impacted during the COVID-19 pandemic compared to the same timeframe in 2019. Using retrospective data, we found 27 significant associations between procedure delay and adverse patient outcomes. Clinician review indicated that 88.9% of the significant associations were plausible and potentially clinically significant. Analytic pipelines for this study are available online. CONCLUSION: Our approach enables health systems to identify medical procedures affected by the COVID-19 pandemic and evaluate the effect of delay, enabling them to communicate effectively with patients and prioritize rescheduling to minimize adverse patient outcomes.

Improving Cancer Data Interoperability: The Promise of the Minimal Common Oncology Data Elements (mCODE) Initiative.

PURPOSE: Because of expanding interoperability requirements, structured patient data are increasingly available in electronic health records. Many oncology data elements (eg, staging, biomarkers, documentation of adverse events and cancer outcomes) remain challenging. The Minimal Common Oncology Data Elements (mCODE) project is a consensus data standard created to facilitate transmission of data of patients with cancer. METHODS: In 2018, mCODE was developed through a work group convened by ASCO, including oncologists, informaticians, researchers, and experts in terminologies and standards. The mCODE specification is organized by 6 high-level domains: patient, laboratory/vital, disease, genomics, treatment, and outcome. In total, 23 mCODE profiles are composed of 90 data elements. RESULTS: A conceptual model was published for public comment in January 2019 and, after additional refinement, the first public version of the mCODE (version 0.9.1) Fast Healthcare Interoperability Resources (FHIR) implementation guide (IG) was presented at the ASCO Annual Meeting in June 2019. The specification was approved for balloting by Health Level 7 International (HL7) in August 2019. mCODE passed the HL7 ballot in September 2019 with 86.5% approval. The mCODE IG authors worked with HL7 reviewers to resolve all negative comments, leading to a modest expansion in the number of data elements and tighter alignment with FHIR and other HL7 conventions. The mCODE version 1.0 FHIR IG Standard for Trial Use was formally published on March 18, 2020. CONCLUSION: The mCODE project has the potential to offer tremendous benefits to cancer care delivery and research by creating an infrastructure to better share patient data. mCODE is available free from www.mCODEinitiative.org. Pilot implementations are underway, and a robust community of stakeholders has been assembled across the oncology ecosystem.

Lung Cancer Detection using Co-learning from Chest CT Images and Clinical Demographics.

Early detection of lung cancer is essential in reducing mortality. Recent studies have demonstrated the clinical utility of low-dose computed tomography (CT) to detect lung cancer among individuals selected based on very limited clinical information. However, this strategy yields high false positive rates, which can lead to unnecessary and potentially harmful procedures. To address such challenges, we established a pipeline that co-learns from detailed clinical demographics and 3D CT images. Toward this end, we leveraged data from the Consortium for Molecular and Cellular Characterization of Screen-Detected Lesions (MCL), which focuses on early detection of lung cancer. A 3D attention-based deep convolutional neural net (DCNN) is proposed to identify lung cancer from the chest CT scan without prior anatomical location of the suspicious nodule. To improve upon the non-invasive discrimination between benign and malignant, we applied a random forest classifier to a dataset integrating clinical information to imaging data. The results show that the AUC obtained from clinical demographics alone was 0.635 while the attention network alone reached an accuracy of 0.687. In contrast when applying our proposed pipeline integrating clinical and imaging variables, we reached an AUC of 0.787 on the testing dataset. The proposed network both efficiently captures anatomical information for classification and also generates attention maps that explain the features that drive performance.

The Impact of Big Data Research on Practice, Policy, and Cancer Care.

The concept of "big data" research-the aggregation and analysis of biologic, clinical, administrative, and other data sources to drive new advances in biomedical knowledge-has been embraced by the cancer research enterprise. Although much of the conversation has concentrated on the amalgamation of basic biologic data (e.g., genomics, metabolomics, tumor tissue), new opportunities to extend potential contributions of big data to clinical practice and policy abound. This article examines these opportunities through discussion of three major data sources: aggregated clinical trial data, administrative data (including insurance claims data), and data from electronic health records. We will discuss the benefits of data use to answer key oncology practice and policy research questions, along with limitations inherent in these complex data sources. Finally, the article will discuss overarching themes across data types and offer next steps for the research, practice, and policy communities. The use of multiple sources of big data has the promise of improving knowledge and providing more accurate data for clinicians and policy decision makers. In the future, optimization of machine learning may allow for current limitations of big data analyses to be attenuated, thereby resulting in improved patient care and outcomes.

Evaluating phecodes, clinical classification software, and ICD-9-CM codes for phenome-wide association studies in the electronic health record.

OBJECTIVE: To compare three groupings of Electronic Health Record (EHR) billing codes for their ability to represent clinically meaningful phenotypes and to replicate known genetic associations. The three tested coding systems were the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes, the Agency for Healthcare Research and Quality Clinical Classification Software for ICD-9-CM (CCS), and manually curated "phecodes" designed to facilitate phenome-wide association studies (PheWAS) in EHRs. METHODS AND MATERIALS: We selected 100 disease phenotypes and compared the ability of each coding system to accurately represent them without performing additional groupings. The 100 phenotypes included 25 randomly-chosen clinical phenotypes pursued in prior genome-wide association studies (GWAS) and another 75 common disease phenotypes mentioned across free-text problem lists from 189,289 individuals. We then evaluated the performance of each coding system to replicate known associations for 440 SNP-phenotype pairs. RESULTS: Out of the 100 tested clinical phenotypes, phecodes exactly matched 83, compared to 53 for ICD-9-CM and 32 for CCS. ICD-9-CM codes were typically too detailed (requiring custom groupings) while CCS codes were often not granular enough. Among 440 tested known SNP-phenotype associations, use of phecodes replicated 153 SNP-phenotype pairs compared to 143 for ICD-9-CM and 139 for CCS. Phecodes also generally produced stronger odds ratios and lower p-values for known associations than ICD-9-CM and CCS. Finally, evaluation of several SNPs via PheWAS identified novel potential signals, some seen in only using the phecode approach. Among them, rs7318369 in PEPD was associated with gastrointestinal hemorrhage. CONCLUSION: Our results suggest that the phecode groupings better align with clinical diseases mentioned in clinical practice or for genomic studies. ICD-9-CM, CCS, and phecode groupings all worked for PheWAS-type studies, though the phecode groupings produced superior results.

Advances in website information resources to aid in clinical practice.

The World Wide Web, which has been widely implemented for roughly two decades, is humankind's most impressive effort to aggregate and organize knowledge to date. The medical community was slower to embrace the Internet than others, but the majority of clinicians now use it as part of their everyday practice. For the practicing oncologist, there is a daunting quantity of information to master. For example, a new article relating to cancer is added to the MEDLINE database approximately every 3 minutes. Fortunately, Internet resources can help organize the deluge of information into useful knowledge. This manuscript provides an overview of resources related to general medicine, oncology, and social media that will be of practical use to the practicing oncologist. It is clear from the vast size of the Internet that we are all life-long learners, and the challenge is to acquire "just-in-time" information so that we can provide the best possible care to our patients. The resources that we have presented in this article should help the practicing oncologist continue along the path of transforming information to knowledge to wisdom.